Representing Dynamics of Facial Expressions

Motion capture (mocap) is widely used in a large number of industrial applications. Our work offers a new way of representing the mocap facial dynamics in a high resolution 3D morphable model expression space. A data-driven approach to modelling of facial dynamics is presented. We propose a way to combine high quality static face scans with dynamic 3D mocap data which has lower spatial resolution in order to study the dynamics of facial expressions. 2 Static 3D Face Expression Model Space Static Capture We used a 3DMD 3D face scanner to collect a set of high quality 3D face scans of a volunteer. The set of expressions used here was: neutral, happiness, anger, fear, surprise, sadness and disgust. Resulting meshes consist of 15000 vertexes. Dense non-rigid 3D Face Registration We used a dense non-rigid 3D face registration software tool [3] to bring all 3D meshes into correspondence. The resulting 3D meshes consist of 3300 vertexes. 3D Morphable Model of Facial Expressions Based on the registration results we generated a 3D morphable model of facial expressions [1]. Every 3D expression is mapped to a point within the coordinates of the PCA based model space. 3 Representation of the Facial Dynamics Motion Capture Data The facial dynamics was recorded at 120Hz using a marker-based motion capture system for a subject performing a set of expressions. The sequence consisted of 3113 frames, containing 3D positions of 66 markers that were placed on the actors face. Re-targeting of Mocap Data Mocap data was poseand scale-normalised and re-targeted to an identity from the static dataset [2]. The resulting sequence was re-targeted to the face presented in the 3D Static Database. We applied Radial Basis Functions (RBF) to interpolate the mocap data to 3D Meshes using the method from [3]. Fig. 2 illustrates used approach. Mapping to expression space We then project the RBF interpolated data into the 3D morphable model expression space. The resulting trajectory path in the morphable Model Space was used to study natural human facial dynamics. Fig.1. shows mocap trajectory path in the model space plotted along the first three principal components Fig.1 Facial dynamics of mocap data presented in PCA space 4 Conclusions This paper used a linear statistical method (PCA) to represent the motion capture facial dynamics with sparse 3D points into a relatively high-resolution 3D morphable model expression space constructed from static scans. This provides a representation of the dynamic expression trajectory in the expression space and allows reconstruction of a highresolution expression sequence. Further work is required to evaluate this approach as a method for retargeting motion capture data to animate high-resolution models and to model the facial dynamics during expression for realistic animation. Figure 2: Retargeting mocap data to high-resolution model AcknowledgementsThe authors are grateful to EPSRC for the Grant EP/C53879XDynamic Faces: Understanding the Dynamics of Real Faces. References[1] V.Blanz and T.Vetter. A morphable model for thesynthesis of 3D faces. In SIGGRAPH, pp. 187 --194, August1999.[2] M. Sanchez, J.Edge, S.King and S.Maddock. Use and Re-use of Facial Motion Capture Data. In Vision, Video andGraphics, July 2004[3] J.R. Tena et al. Dense non-rigid 3D Face Registration. Toappear in AVSS, November 2006.